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ABSTRACT 


The LINCS Storage System (LSS) has been in 
production at Lawrence Livermore National 
Laboratory since January, 1988. In the devel- 
opment of the LSS, the designers made key 
architectural decisions which included the 
separation of data and control messages, a 
network-wide locking mechanism, and the 
separation of the naming mechanism from 
other object managers. Other important issues 
of the system deal with space management, 
descriptor management, and system adminis- 
tration. This paper outlines the LSS software 
system and then focuses on these key design 
decisions and issues with the intent of providing 
some insight into the types of difficulties 
designers face in developing a distributed 
storage system. 


OVERVIEW OF THE SOFTWARE SYSTEM 


The LINCS. Storage System (LSS) is based on 
the IEEE Mass Storage Reference Model! and 
integrates host, central, and archival storage 
systems into one transparent, logical system 
(see Figure 1). The host system storage media 
consists of solid state and rotating disks on lo- 
cal machines. The central system medium is a 
collection of magnetic disks on the storage ma- 
chine. The archival system medium is 
cartridge tape kept either in on-line robotic 
devices or in off-line vaults. 


The software components of the LSS were de- 
signed to operate as servers on a distributed 


network.2 Their design extends the classic 
client-sever model through the use of multitask- 
ing within servers and clients to support con- 
current access to objects. The two main types of 
LSS servers are name servers, which translate 
human-oriented names to machine-oriented ob- 
ject identifiers, and bitfile servers, which man- 
age access to bitfiles on disk or on archival me- 
dia (currently model 3480 tape cartridges). Host 
machines and the storage computer have name 
servers that manage name/object identifier 
pairs, bitfile servers that manage bitfiles on the 
various storage media, and bitfile movers that 
move bitfiles between archival tape and central 
disk and between host bitfile servers and the 
central server.** 


In the remainder of this section we briefly out- 
line the abstract objects managed by the storage 
system and the major components of the system. 
The two main abstract objects of the LSS are the 
bitfile, implemented by the bitfile servers, and 
the directory, implemented by the name servers. 
A bitfile is viewed as a descriptor and a body. 
The descriptor contains attributes of the bitfile, 
such as the time the bit segment was last written 
and the bitfile size. The body is a sequence of 
uninterpreted, logically contiguous bits. The 
operations that can be performed on the body 
include functions such as create, destroy, read, 
and write. Fields of the bitfile descriptor can be 
read with the interrogate operation and, when 
allowed, written with the change operation. A 
directory is viewed as a descriptor and a list of 
entries. The descriptor contains attributes of the 
directory such as the time an entry was last 
inserted or deleted and the number of entries 
contained in the directory. An entry is a 
human-name/object-identifier pair. Directo- 
ries may be created, destroyed, or listed and 


** In Figure 1, the bitfile movers are repre- 
sented by the arrows between the servers. 
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entries may be created, deleted, fetched or 
renamed. As with the bitfile descriptor, fields 
in the directory descriptor may be read with the 
interrogate operation and written with. the 
change operation. 


The name servers provide a transparent nam- 
ing service based.on the use of unique, network- 
wide object identifiers for all resources. Object 
identifiers in the system have the same struc- 
ture regardless of the type of object they identify. 
This allows objects of varying types to be cata- 
loged in the same directory, providing a uni- 
form naming mechanism across all objects. 
Since the directory object identifiers are also 
globally unique and can be stored in directo- 
ries, a logically single, hierarchical, directed- 
graph directory structure can be built. This 
structure supports name transparency; transla- 
tion of a pathname initiated from any site in the 
network always references the same object. 
Furthermore, there is no necessary relationship 
between the directory location and the location of 
the objects cataloged in the directory. To help 
reduce network delay when accessing a direc- 
tory, it is planned that the host name servers 
- will cooperate with the central name server to 
cache and migrate directories. 


The bitfile servers provide network-wide ran- 
dom access to bitfiles. The design model for the 
integrated network bitfile system is to have 
- fully integrated host, central, and archival bit- 
file servers and movers which support auto- 
matic migration throughout the entire storage 
hierarchy. The connection between the central 
and host bitfile servers is not yet fully imple- 
mented. However, the current bitfile system 
hierarchy is integrated from the central bitfile 
server through the archive's off-line vault vol- 
umes. When the bitfile system is fully inte- 
grated, a client on a host will request a bitfile 
access from his local server. If the local server 
has no knowledge of the bitfile being requested it 
will contact the central server. The central 
server will find the bitfile, which may reside on 
another host machine or on central media or in 
the archive, and send a copy to the requesting 
host bitfile server. Until the hierarchy is fully 
connected, a client on a host machine may ac- 
cess a bitfile managed either by the central or by 
the archival bitfile server through direct com- 
munication with the central server or by invok- 


ing a utility that copies the bitfile to a local bit- 
file server. : 


The central and archival levels of the storage 
hierarchy are integrated, using a 200-gigabyte 
disk cache as a front-end to tape. As new bitfiles 
accumulate, they are automatically written to 
tape. The descriptors for all bitfiles on tape are 
maintained on disk for easy access. On a read 
request, a bitfile is automatically cached from 
tape to the disk cache. Bitfiles are purged from 
the disk cache using a bitfile-size—time-of-last- 
access algorithm. 


Using this overview as background, we now fo- 
cuses on several key design decisions and is- 
sues. 


SEPARATION OF CONTROL AND DATA 


The separation of control and data messages in 
the LSS improves transfer rates and allows 
third-party copies during which data does not 
pass through bitfile server buffers. Data asso- 
ciated with reads or writes is received or sent on 
a data communication association (source- 


destination address pair) separate from the con- 


trol communication associations; see Figure 
2a. The third-party copy model is illustrated in 
Figure 2b. This design can create a pipe-line of 
services as shown in Figure 2c. 


The optimization of third-party copies is sup- 
ported by receive-any and send-any mecha- 
nisms in the LINCS message-system interface. 
These mechanisms use association capabilities 
called stream numbers to make communica- 
tions secure; associations are defined by the 
triple (source address, destination address, 
stream number). The receive-any and send- 
any mechanisms allow a process to declare that 
it is willing to receive or send messages with 
one or more members of the triple unspecified— 
i.e., any source, or any destination, or. any 
stream number. For example, the third-party 
copy protocol uses the receive-any by indicating 
a specific destination address and a specific 
stream number. but an unspecified source ad- 
dress; a rendezvous occurs when a message ar- 
rives matching the receive-any's specific desti- 
nation address and specific stream number. 
The third-party copy protocol uses the send-any 
by indicating a specific source address and a 
specific stream number but an unspecified des- 


tination address. The message is flow-blocked 
at the source machine until an ack with the spe- 
cific source address and stream number is re- 
ceived by the source machine, completing the 
rendezvous by matching the acknowledging 
destination address with the any-destination 
address. 


The third-party copy utilizes these mechanisms 
in the following protocol. When a controlling 
process requires data to be moved between sec- 
ond and third processes, it first sends a message 
to the second, over a control association, that in- 
cludes: an indication that the process is to use 
the send- or receive-any mechanism in the data 
transfer, an indication whether the process will 
send or receive data, and a stream number. In 
response, the second process will: 1) invoke a 
receive-any, if receiving data, or a send-any, if 
sending data, using the stream number re- 
ceived from the controlling process and 2) send 
a message back to the controlling process con- 
taining its local data address used in invoking 
the receive- or send-any. The controlling pro- 
cess then sends a message to the third process 
that includes: an indication that the process is to 
use a send- or receive-specific in the data trans- 
fer, an indication whether the process will send 
or receive data, the data address received from 
the second process, and the stream number. In 
response, the third process invokes a receive- 
specific, if receiving data, or a send-specific, if 
sending data, using the data address and 
stream number received from the controlling 
process. The any message of the second process 
will rendezvous with the specific data message 
or specific ack of the third process and the data 
will flow directly between these processes. 
When the data transfer is complete both source 
and sink processes send a message to the con- 
trolling process acknowledging completion of 
the transfer (see Figure 3). 


The LSS utilizes bitfile movers to actually 
transfer the data. A bitfile mover performs data 
transfer directly between channels such as 
memory or networks or devices such as disk or 
tape. In our UNIX implementations, the bitfile 
movers have been implemented as either nor- 
mal user processes or as_ kernel entities, the 
former for portability and the latter for perfor- 


mance. All bitfile movers use the same com- 
munication and lightweight tasking libraries, 
whether in kernel or user space. On a single 
machine, data is transferred by the mover be- 
tween a device and application memory; over a 
network, data is actually moved directly be- 
tween bitfile movers. For a full discussion of 
the bitfile movers, see reference 3. 


Because the LSS separates data and control mes- 
sages and utilizes the mover processes, the bit- 
file servers never touch the sequence of bits in 
the bitfile body. This not only avoids data copy- 


‘ing when only one machine is involved in a 


transfer, but it is also important when a pro- 
gram on machine A invokes a transfer of all or 
part of a bitfile from machine B to machine C. 
The program on machine A controls the move- 
ment of the data but the data flows directly from 
machine B to machine C. This architecture 
gives designers the flexibility to place data 
movement control on lower-performance ma- 


- chines without impacting the performance of the 


system. 


The LSS experience with separation of data and 
control messages has shown this mechanism to 
be valuable for both performance and modular- 
ity. 


ARCHIVAL SPACE MANAGEMENT 


Storage technology is failing to keep pace with 
the rapid growth of supercomputer memory ca- 
pacities. It has been the experience at Lawrence 
Livermore National Laboratory (LLNL) that a 
small number of large bitfiles can occupy a 
large part of the system's storage resources. 
These bitfiles are the results of scientific simu- 
lations and their sizes are proportional to super- 
computer memory size. The capacity of new 
robotic libraries, such as the STC 4400, once 
thought to be adequate, will soon be insufficient 
to handle the data produced at a large scientific 
laboratory. To fully utilize available on-line 
robotic devices, the LSS completely fills each 
tape cartridge by writing data until an end-of- 
file error is received. A tape may hold many 
files or a file may span many tapes. The LSS 
also manages the on-line archive as a level in 
the storage hierarchy. Inactive bitfiles migrate 
to lower-level, off-line vault volumes when on- 
line archival space is needed; bitfiles in the 


vault are moved up to the on-line robotic devices 
as they are accessed. 


Algorithms used to manage space on magnetic 
disks cannot be used to manage archival media 
that do not provide random write access. To 
reuse free space on a magnetic tape the active 
data on the tape must first be copied to a new vol- 
ume, an operation called repacking. Since 
copying is an expensive operation, only vol- 
umes with a considerable ‘amount of free space 
should be repacked. 


The necessity of repacking to reclaim unrefer- 
enced space affects the criteria by which LSS bit- 
files are chosen to migrate to the vault. While it 
is desirable to use a space-time product algo- 
rithm that allows smaller bitfiles to remain 
higher in the hierarchy longer than larger bit- 
files, it is not desirable to migrate a few short 
bitfiles from many robotic tape cartridges to 
vault cartridges if the space released by these 
bitfiles does not make any of the robotic tapes 
candidates for repacking. Only bitfiles that to- 
gether account for enough free space on their 
volumes should be migrated to the vault. 


In the LSS, the archival server maintains a 
table of the current number of referenced blocks 
on each archival volume. The server: main- 
tains a minimum number of free volumes by 
repacking volumes on which less than half of 
the data blocks are referenced. If there are not 
enough free blocks on candidate volumes, it mi- 
grates bitfiles to the vault according to a space- 
time algorithm, except that bitfiles are migrated 
only if they result in volumes becoming candi- 
dates for repacking. Migrating bitfiles rather 
than volumes to the vault minimizes the number 
of active bitfiles in the vault and minimizes the 
number of cartridges handled manually 
(particularly convenient since LLNL's vault 
and on-line facilities are located in different 
buildings). 


ARCHIVAL FILE DESCRIPTOR 
MANAGEMENT 


In the LSS, bitfile descriptors for bitfiles man- 
aged by the archival server are kept on dedi- 
cated, redundant disk for fast queries and up- 
dates and to add flexibility in assigning the 
physical location of the bitfile body. Because the 
bitfile descriptors are vital to the system, great 


care is taken to ensure their integrity and re- 
covery. 


To ensure the integrity of the bitfile descriptors, 
atomic transactions are used to perform updates 
to the duplicated disks. To protect against mul- 
tiple disk failures, a log of all descriptor inser- 
tions, deletions, and modifications is ::main- 
tained. Sixteen descriptor updates (contents of 
one physical disk sector) are collected in an in- 
ternal buffer before being atomically written to 


disk. The buffer. is also flushed to disk every 


two minutes if not enough updates have been col- 
lected. The log is also written to tape periodi- 
cally to guard against the loss of the log disks. 
Since recovery after many months of updates 
from an incremental log would be slow, a com- 
plete tape backup of all the archival server disks 
is performed weekly. To recover from a catas- 
trophic failure, the incremental log from the last 
week is applied to the latest full backup. 


File descriptor management in the LSS ensures 
fast access and data integrity at a moderate cost. 
In twenty-five years of operation of the LSS and 
the predecessor system, no failures occurred that 
required accessing the backup tapes. 


FILE SYNCHRONIZATION 


To preserve a bitfile’s consistency when it is ac- 
cessed concurrently by multiple applications, 
the LSS uses a distributed locking mechanism 
with notification instead: of lifetime timeout; an 
application maintains a lock until it is finished 
accessing the bitfile or it is notified that another 
application is interested in accessing the bitfile. 
When an application receives a notification it 
may release the lock and allow access to the bit- 
file by the other application or it may refuse to 
give up the lock. An application might reason- 
ably keep bitfiles locked for months if no other 
application wishes to access them, so locks do 
not timeout. If an application holding a lock 
does not respond to a request to release the lock, a 
lock-breaking mechanism can be used by the 
requesting application to recover the lock. 


The LSS locking mechanism is based on classic 
read/write locks,4 with extensions for notifica- 
tion and lock breaking. A lock held by a user 
application may span.many read .and write ac- 
cesses, whereas the bitfile servers lock objects 
only for individual accesses. There are three 


_ types of locks: no-lock, read-lock and write- 
lock. A write-lock allows both reading and 
writing by the client holding the lock. There 
may be multiple concurrent read-locks on a bit- 
file allowing multiple readers, but a write-lock 
excludes all other readers and writers. Locks 
may be categorized into levels, with the highest 
level being a write-lock and the lowest level be- 
ing a no-lock. Each bitfile server manages 
locks for its own clients. 


The lock operations on a bitfile include set-lock, 
reduce-lock and break-lock. Set-lock is used by 
clients either to set a lock or to lower the level of 
a lock on a bitfile. Reduce-lock is used to ask a 
client to lower or release its lock on a bitfile. A 
client sends a break-lock request to a bitfile 
server to break a lock held by a client that is 
down. The granularity of locking is a bitfile. A 
lock request includes a bitfile identifier, a lock 
type, and a notification address, which is the re- 
questing client's address in the case of a set- 
lock, and the lock holder's address as well in 
the case of a reduce- or break-lock. The notifi- 
cation address in a lock is used to identify the 
lock holder. 


Crash recovery considerations for locking in- 
clude: 1) how to proceed when a bitfile server is 
unavailable; and 2) how to restore lock state 
when a bitfile server reinitializes after a crash. 
If a bitfile server fails to respond to a reduce lock 
request, causing the requesting bitfile server to 
refuse an application access to a bitfile, then the 
application may send a break lock request. In 
the LSS, breaking a lock invalidates changes 
that were made while the lock was in effect. A 
client would choose this action only when losing 
updates is preferable to waiting for the bitfile 
server to return to service. 


ADMINISTRATIVE REQUIREMENTS OF A 
DISTRIBUTED FILE SYSTEM 


We have identified six administrative require- 
ments for the LSS: 1) to ensure fair use of stor- 
age, 2) to ensure efficient use of storage, 3) to 
achieve high performance, 4) to minimize cost, 
5) to provide for accountability of storage re- 
sources, and 6) to permit cost recovery. The pro- 
posed method for meeting these requirements is 
a combination of charging and allocation. 
Charging refers to a mechanism by which users 
are charged for the storage resources (space and 


transfers) they consume. Allocation refers to a 
mechanism for ensuring that each user or group 
of users has an appropriate share of the avail- 
able storage resources.5 In this section, we will 
focus on the difficulty of designing mecha- 
nisms that are consistent with the administra- 
tive requirements and with the design goal of 
location transparency’ and with user expecta- 
tions. 


The predecessor system to the LSS, which con- 
trolled only central and archival storage, 
lacked adequate administrative measures... It 
allowed unrestricted, free use of storage, and 
provided no incentive for users to restrict: the 
amount of data they generated, or to delete un- 
wanted data. On the host disks, storage was also 
free, but an attempt was made to restrict the 
amount of data on disks by purging idle bitfiles. 
This measure proved to be ineffective. To pre- 
vent their bitfiles from being purged, users sim- 
ply ran applications that periodically touched 
their bitfiles to keep them active. 


To dispel the notion of free storage, a new charg- 
ing policy was implemented in the early days of 
the LSS. The motivation behind this initial pol- 
icy was to recover the cost of the system and to 
discourage users from maintaining all of their 
bitfiles on expensive fast-access storage. The 
algorithm for computing charges was [bitfile- 
length times bitfile-age times charge-rate], 
where the charge rate varied from one storage 
device to another. The bitfile age was the 
smaller of the time since last charged or the 
time since creation. The information needed to 
compute the charge was collected weekly as each 
bitfile server traversed its descriptors. A sepa- 
rate utility computed charges based on the re- 
ports from each LSS server, decremented user 
bank accounts accordingly, and produced 
billing reports. 


The problem with this scheme is that the same 
bitfile is multiply charged if it happens to reside 
on more than one bitfile server. This is the case, 
for example, when a central bitfile is cached on 
a host disk. The appearance of multiple charges 
for the same bitfile, and of charges that reflect 
the location of bitfiles in the system, clearly de- 
feats the system design goal of transparency. 
Mechanisms to remedy this problem are being 
considered. One such mechanism allows users 
to classify bitfiles as active, archival, etc., to 


help them control their costs and to improve the 
effectiveness of migration algorithms. We are 
also considering a fixed rate that is independent 
of the storage medium. As an interim fix, the 
system has been changed to charge only for bit- 
files stored on archival. volumes. 


Charging alone does not meet all of the admin- 
istrative requirements. Because there are phys- 


ical limits to a storage system and because — 


purely economic factors do not seem to be ade- 


quate, we are considering allocation | limits to | 
ensure fair sharing of storage resources and to 


ensure good performance of the system. The 
proposal we have adopted, though not yet imple- 
mented, includes two types of allocation, global 
and disk. Global allocation applies to the total 
space acquired at all levels of storage, includ- 
ing host disk, central disk, and archival tape. 
Each user has a global allocation, which is an 
upper limit on the total amount of data a user 
may store. Once this limit is reached, the user 
may not create more bitfiles, or extend existing 
bitfiles, until he generates free space. by. destroy- 
ing bitfiles. 


Disk allocations apply only to host and central 
disk. . Users have maximum disk: allocations 
but are allowed to exceed their allocations if 
space allocated to other users is not in use. 
When space is needed, bitfiles belonging to 
users who have exceeded their allocations are 
migrated first. For. host disk, users also have 
minimum disk allocations. Users’ bitfiles will 
not migrate if their space utilization falls below 
this amount. 


Designing adequate charging and allocation 
schemes for a transparent system poses a diffi- 
cult challenge. Charging schemes that strive to 
do accurate cost recovery are inherently incon- 
sistent with the goal of transparency. There is 
much to learn about selecting and implement- 
ing effective niminiettative Policies in these 
areas. . 


NETWORK-WIDE NAMING acai : 
AND USER EXPECTATIONS 


Before we implemented a network-wide nam- 
ing mechanism, the LSS was a distributed sys- 
tem but nota transparent one. Users were aware 
of object location and had to explicitly invoke 
transfer routines to move bitfiles. between host 


machines and the central server. Inactive. host 
bitfiles were automatically destroyed, and ac- 
cess to archival bitfiles was relatively slow. 
The central name server and each host main- 
tained separate, unconnected directory struc- 
tures which did not migrate between machines. 
As a result of this environment, users wrote pro- 
grams to keep bitfiles on host disks by periodi- 
cally accessing bitfiles stored on the hosts. 
Users resisted the idea of location transparency 


_ because it meant they would fete eentrel of ace 
cess. time. eon 


Location. dransuatenny was aitierea in two 
phases. The first included creation of a net- 
work-wide directory structure, for name trans- 
parency; and the second implemented caching 
and migration of directories, for performance. 
In the first phase, we. connected the directory 
structures stored on the host machines with the 
directory structure in central. Central and 
archival bitfiles and central directories could 
then be cataloged in directories on the host ma- 
chines. and vice versa, creating a transparent, 
eross-machine naming structure. Because bit- 
files and directories have globally unique 
identifiers, the storage system could locate the 
resources regardless of the directory in which 
they were cataloged and regardless of which 
server managed the resource. 


In this strange, new storage world, users be- 
came. frustrated when they unknowingly 
crossed machine. boundaries in perusing 
through their directory structures because per- 
formance suddenly declined. Yet, many. re- 
sisted the idea of caching and migration of bit- 
files. and directories to improve..performance; 
these users desired to control location of their 
bitfiles and directories to ensure fast access. 
We believe, however, that the system can best 

manage object location, much as demand pag- 


ing eaten manage’ virtual memory. 


There. are. two significant lessons to be learned 


from our experience. First, when creating a 
transparent storage system it is important to 
achieve transparency in all. aspects before 
putting the system on-line. Second, in present- 
ing a new storage system to users, system de- 
signers need to carefully consider what the 
users expect and educate them about the differ- 
ences in the new system. 


SEPARATION OF HUMAN NAMING FROM 
OTHER OBJECT SERVERS 


When considering a human-oriented naming 
- mechanism for a distributed storage system, de- 
signers must choose between one that is integral 
to the object managers, such as UNIX and CFS, 
and one that is isolated in separate name man- 
agers, such as XDFS and ALPINE.® We chose 
the latter design, recognizing that it has both ad- 
vantages and disadvantages. The advantages 
of a separate naming mechanism include: 
1) providing a uniform mechanism for naming 
many different types of objects, not just bitfiles; 
2) providing all other servers independence 
from human-oriented naming conventions, al- 
lowing them to function in a variety of user en- 
vironments;? 3) permitting the other servers to 
be optimized to manage their own objects without 
the need to deal with human-oriented naming 
issues;8 4) permitting new types of objects to be 
named in the same way as existing objects, fa- 
cilitating extensibility;? 5) permitting several 
forms of application-dependent and general- 
purpose higher-level name services te be pro- 
vided in addition to a particular directory ser- 
vice;8 and 6) permitting applications to create, 
access, and destroy objects. without storing the 
object identifiers in any name service.9 


The disadvantages of a separate naming mech- 
anism, discussed below, include: 1) compli- 
cating certain aspects of storage management; 
2) requiring additional security mechanisms; 
and 3) causing performance degradation in a 
particular class of applications. 


We believe that the advantages of a separate 
naming mechanism outweigh the disadvan- 
tages and that the LSS has achieved those advan- 
tages. In particular, a separate naming mech- 
anism allows the bitfile servers of the LSS to be 
integrated with the naming mechanism of any 
existing operating system. We are now work- 
ing on minimizing the effects of its disadvan- 
tages, as described below. 


Storage Management 


The LSS servers will perform functions given a 
valid object identifier containing the appropri- 
ate access rights. The general availability of 
certain server functions, in combination with 
the separation of the name servers from the other 


object servers, has consequences for storage 
management. Specifically, clients have the po- 
tential of creating lost objects, objects for which 
there are no extant identifiers, and of creating 
dangling pointers, identifiers no longer. point- 
ing to valid objects.. Lost objects are created 
when a client deletes all references to the object 
without destroying the object. Dangling point- 
ers are created when a client destroys an object 
but does not delete all the references to it. 


The problems of lost objects and dangling point- 
ers can be solved by correctly managing refer- 
ence counts and by prohibiting explicit destroys 
of objects. A count of all references to it is kept 
with each object as part of its storage manage- 
ment state. When the count becomes zero, 
meaning the object is no longer referenced, the 
space it occupies can be reclaimed for further 
use. All applications in the LSS environment 
that store identifiers to objects need to increment 
and decrement reference counts correctly to 
protect against lost objects and delete object 
references before destroying objects to protect 
against dangling pointers. For example, name 
servers should send increment and decrement 
messages to the object servers when objects are 
inserted and deleted from directories. When 
the reference count goes to zero, the object can be 
implicitly destroyed. 


However, the increment- and decrement- 
reference-count functions are not controlled by 
the current LSS access control mechanism and 
therefore may be invoked by any client possess- 
ing a valid machine-oriented identifier. 
There is no way to ensure that clients will in- 
voke the reference-count functions correctly. 
Furthermore, there is no way to guarantee that 
all naming services or applications that store 
object identifiers will correctly increment and 
decrement the reference counts as objects are in- 
serted and deleted from their databases. 
Similarly, there is no way to guarantee that all 
object references are deleted before the object is 
explicitly destroyed.. In these circumstances, 
reference counts cannot be relied upon to re- 
claim storage space. 


One solution to these problems is to allow only 
trusted naming applications to invoke the incre- 
ment- and decrement-reference-count and. de- 
stroy functions and to require clients to store 
machine-oriented identifiers only in these ap- 


plications. The additional access control nec- 
essary to implement trusted applications could 
be obtained through several mechanisms in- 
cluding rights amplification capabilities, rights 
verification servers, or access lists. The LSS 
name servers would be the initial set of trusted 
applications, but it: would be reasonable to in- 
clude any naming service that could establish 
its correct use of reference-count and destroy 
functions. Of course, a requirement to use a re- 
stricted set of applications to catalog object iden- 
tifiers would restrict clients’ ability to use their 
own naming services. 


Security 


Security is affected in two ways by the separate 
naming mechanism. . First, it is hampered 
when a client, possessing a valid object identi- 
fier, maliciously or inadvertently destroys an 
object still being referenced by other clients by 
sending repeated decrement-reference-count 
functions to the object server. Likewise, too 
many increment requests can prevent an ob- 
ject's implicit destruction, leaving the object 
available for continued access after it should 
have been destroyed. This problem is solved by 
the same trusted-naming-application mecha- 
nism suggested above that solves the storage 
management problems. 


Second, in an-environment with a separate 
naming mechanism, ‘clients ‘can — obtain, 
through several server functions, a machine- 
oriented object identifier for use as parameters 
in subsequent function requests to the servers 
(e.g., read and write). Because clients can pos- 
sess machine-oriented identifiers, there must be 
security mechanisms to prevent the client from 
changing the identifier to another valid identi- 
fier and to protect identifiers against the threats 
of forgery, theft, and reuse. In the LSS, the ma- 
chine-oriented identifiers are encrypted by 
servers using a DES10 cryptographic checksum, 
to prevent client tampering and to protect 
against forgery.. Encryption algorithms exist 
which also protect against theft and reuse,1! but 
they are not used in the LSS because they were 
determined to be unnecessary in LLNL's secure 
physical environment. Because the LSS does 
not protect against theft and reuse, users must 
exercise care in storing the machine-oriented 
identifiers: storing indentifiers in trusted 


name servers is safe; storing them on worksta- 
tions may. be unsafe. 


Performance 


Separation of the naming mechanism from the 
other object servers has both a positive and a 
negative impact on performance. Separation of 
human naming from other object. servers has 
hada positive impact on performance through 
modularity. The modularity gained from sepa- 
rating the naming mechanism from the object 
servers is observed in two important areas. 


_ First, software errors pertaining directly to the 


name translation code do not affect the object 
servers. Additionally, hardware failures af- 
fecting a given name server do not disturb other 
object servers on remote machines. If a name 
server were to go down for a period of time due to 
either of these causes, all applications which had 
previously translated their human-oriented 
names into machine identifiers could continue 
processing. Moreover, if the period of down time 
for. the name. server. were. short, an application 
may not even notice that the naming service had 
been interrupted: 


The second modularity advantage of a separate 
naming mechanism is the efficiency gained in 
other object servers by being independent of 
human-oriented naming conventions. The 
name server acts as an intermediary between 
the human world and machine processes. No 
other object server need contain algorithms 
which parse the strings of human names or 
package them for passing. in. messages. 
Instead, the object servers deal only with 
machine-oriented identifiers which can be 
easily interpreted and utilized. 


However, performance suffers when an applica- 
tion requires the attributes of the objects con- 
tained in a directory. This type of application, 
such as the UNIX Is utility, first must contact the 
name server to obtain a list of machine-oriented 
object identifiers. Because the name server does 
not contain other information about the objects, 
the application must then contact the object 
servers to obtain the desired attributes. Since the 
name server has been generalized to store ob- 
jects of many different types, the application 
may have to contact several object servers. 
Performance improvement techniques to limit 
this disadvantage are being designed and im- 


plemented. For example, the caching and mi- 
gration of objects will help improve perfor- 
mance by causing both directories and other ob- 
jects to be locally cached, resulting in faster ac- 
cess times. 


The separation of the naming mechanism from 
the other object servers in the LSS has given the 
system and its clients flexibility, extensibility, 
and modularity not found in integral naming 
mechanisms. Although this separation has dis- 
advantages, techniques exist to lessen their im- 
pact. Moreover, the system's flexibility and 
modularity allow it to be used under client inter- 
faces that present the conventional, integrated 
view. In this environment, the user applica- 
tions would not suffer from most of the disad- 
vantages discussed above, yet the system would 
benefit from most of the advantages of a separate 
naming mechanism. 


CONCLUSION 


This paper discusses several key design goals 
and implementation areas of the LSS that sup- 
port its extensibility, its modularity, and its 
flexibility. The separation of data and control 
messages has had a positive impact on perfor- 
mance and modularity by allowing third-party 
copies without the actual passing of data through 
the bitfile servers. Efficient management of 
storage media and bitfile headers has increased 
storage utilization and provided integrity of the 
header information. A network-wide locking 
mechanism has been designed that preserves an 
object's consistency when accessed concur- 
rently by multiple applications. The separation 
of the human-oriented naming mechanism 
from the other object servers has given the sys- 
tem and its clients flexibility, extensibility, and 
modularity not found in an integral naming 
mechanism. 
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